Interaction with a virtual network

ABSTRACT

Systems and method for the management of virtual machine instances are provided. A network data transmission analysis system can host virtual machine networks. A component of a hosted virtual machine network is configured in a manner to receive commands directed towards a simulated network device. The component may then execute a process or processes on the hosted virtual machine network which correspond to the received command.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.12/985,984, entitled INTERACTION WITH A VIRTUAL NETWORK, and filed onJan. 6, 2011, the entirety of which is incorporated herein by reference.

BACKGROUND

Generally described, computing devices utilize a communication network,or a series of communication networks, to exchange data. Companies andorganizations operate computer networks that interconnect a number ofcomputing devices to support operations or provide services to thirdparties. The computing systems can be located in a single geographiclocation or located in multiple, distinct geographic locations (e.g.,interconnected via private or public communication networks).Specifically, data centers or data processing centers, herein generallyreferred to as a “data center,” may include a number of interconnectedcomputing systems to provide computing resources to users of the datacenter. The data centers may be private data centers operated on behalfof an organization or public data centers operated on behalf, or for thebenefit of, the general public.

To facilitate increased utilization of data center resources,virtualization technologies may allow a single physical computing deviceto host one or more instances of virtual machines that appear andoperate as independent computing devices to users of a data center. Withvirtualization, the single physical computing device can create,maintain, delete or otherwise manage virtual machines in a dynamicmatter. In turn, users can request computer resources from a datacenter, including single computing devices or a configuration ofnetworked computing devices, and be provided with varying numbers ofvirtual machine resources.

Generally, physical networks include a number of hardware devices thatreceive packets from a source network component and forward the packetsto designated recipient network components. In physical networks, packetrouting hardware devices are typically referred to as routers, which areimplemented on stand-alone computing devices connected to a physicalnetwork. With the advent of virtualization technologies, networks androuting for those networks can now be simulated using commoditycomputing devices rather than actual routers.

Specifically, in one aspect, individuals, such as system administrators,wishing to manage a physical network may be familiar with specificmethods of interfacing with network devices. For example, a routingdevice may provide a command line interface for administration of therouter's functions. However, simulated routing devices implemented viavirtual instances in a virtual network would not necessarily allow suchinterfaces.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisdisclosure will become more readily appreciated as the same becomebetter understood by reference to the following detailed description,when taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram illustrating an embodiment of a substratenetwork having computing nodes associated with a virtual computernetwork;

FIG. 2 is a block diagram of the substrate network of FIG. 1illustrating logical networking functionality;

FIG. 3 is a block diagram of the substrate network of FIG. 1illustrating a substrate network configuration associated with overlaynetworks;

FIGS. 4A and 4B are block diagrams of the substrate network of FIG. 1illustrating independently determined substrate routing;

FIGS. 5A and 5B are block diagrams of the substrate network of FIG. 1illustrating virtual route selection propagation to the substratenetwork;

FIG. 6 is a block diagram of the substrate network of FIG. 1illustrating the determination of routes into or out of a virtualnetwork by network translation device;

FIG. 7A illustrates a flow diagram for a process of propagating virtualroutes to a substrate network;

FIG. 7B illustrates a flow-diagram for a process of determiningsubstrate routing based on target performance characteristics of theassociated virtual network;

FIG. 8 illustrates a virtual network as simulated by the substratenetwork of FIG. 1;

FIG. 9 is a simplified block diagram of the substrate network of FIG. 1illustrating the processing of commands generated by a client computingdevice; and

FIG. 10 illustrates a flow-diagram for a process of simulating a networkcomponent for facilitating a client computing device interaction.

DETAILED DESCRIPTION

Generally described, aspects of the present disclosure relate to themanagement of virtual machine instances. Specifically, systems andmethods are disclosed which facilitate user interaction with virtualnetworks in a manner similar to interaction with physical networks. Inone aspect, a networking device interface is simulated by a component ofvirtual network. A user may access the simulated networking deviceinterface and interact with the virtual network in such a manner thatthe virtualization of the network is transparent to the user. Forexample, a virtual network may be configured to simulate a physicalnetwork having physical devices connected via a physical router.Illustratively, the function of the simulated router may be implementedwithin the virtual network without the need to simulate the router orwithout any consideration of the actual routing within the virtualnetwork. The user may attempt to send commands to the simulated routerinterface using an expected interface methods, for instance, a commandline interface. In this aspect, an interface component of the virtualnetwork may receive the command and determine the appropriate process orprocesses corresponding to the received command to be executed withinthe virtual network. The interface component may, where appropriate,send a response to the user indicating the results of the command.Illustratively, a user may transmit a command requesting thatstatistical information regarding the operation of the network, such ashop times or packet loss, be displayed. The component may, according tothe command, determine an appropriate process or processes to beimplemented by components of the virtual network in order to obtain thedesired information. The interface component would then return theinformation to the user. In some embodiments, this information may befurther formatted in accordance with the simulated router interface. Asa further illustration, the interface component may serve to allow theuser to alter simulated routing functionality. For example, a user maytransmit a command to the simulated router interface specifying thattraffic from various simulated network components be shaped in aspecified manner. The interface component of the virtual network wouldthen determine an appropriate process or processes to be implemented bycomponents of the virtual network in order to simulate the user'sdesired functionality.

The following section discusses various embodiments of managed networksfor network data transmission analysis. Following that is furtherdiscussion of network data transmission analysis systems and methodsthat can implement management methodologies established by a networkuser.

Managed Computer Networks for Network Data Transmission Analysis

With the advent of virtualization technologies, networks and routing forthose networks can now be simulated using commodity hardware components.For example, virtualization technologies can be adapted to allow asingle physical computing machine to be shared among multiple virtualnetworks by hosting one or more virtual machines on the single physicalcomputing machine. Each such virtual machine can be a softwaresimulation acting as a distinct logical computing system that providesusers with the illusion that they are the sole operators andadministrators of a given hardware computing resource. In addition, asrouting can be accomplished through software, additional routingflexibility can be provided to the virtual network in comparison withtraditional routing. As a result, in some implementations, supplementalinformation other than packet information can be used to determinenetwork routing.

Aspects of the present disclosure will be described with regard toillustrative logical networking functionality for managed computernetworks, such as for virtual computer networks that are provided onbehalf of users or other entities. In at least some embodiments, thetechniques enable a user to configure or specify a network topology,routing costs, routing paths and/or other information for a virtual oroverlay computer network including logical networking devices that areeach associated with a specified group of multiple physical computingnodes. For example, a user (e.g., a network administrator for anorganization) or service provider may configure a virtual or overlaynetwork based on detected events, processing criteria, or upon request.With the network configuration specified for a virtual computer network,the functionally and operation of the virtual network can be simulatedon physical computing nodes operating virtualization technologies. Insome embodiments, multiple users or entities (e.g. businesses or otherorganizations) can access the system as tenants of the system, eachhaving their own virtual network in the system. In one embodiment, auser's access and/or network traffic is transparent to other users. Forexample, even though physical components of a network may be shared, auser of a virtual network may not see another user's network traffic onanother virtual network if monitoring traffic on the virtual network.Logical functionality of a managed computer network may be discussedwith reference to substrate components. In an illustrative embodiment ofthe present disclosure, a substrate component can refer to one or morephysical computing devices which allow the implementation of thediscussed logical functionality. Where reference is made to a substratenetwork, such reference refers to one or more networked substratecomponents.

By way of overview, FIGS. 1 and 2 discuss embodiments wherecommunications between multiple computing nodes of the virtual computernetwork emulate functionality that would be provided by logicalnetworking devices if they were physically present. In some embodiments,some or all of the emulation are performed by an overlay network managersystem. FIGS. 2-4B and 7B discuss embodiments where substrate routingdecisions can be made independently of any simulated routing in theoverlay network, allowing, for example, optimization of traffic on thesubstrate network based on information unavailable to a virtual networkuser. FIGS. 5A-7A discuss embodiments where routing decisionsimplemented on the virtual or overlay network are propagated to thesubstrate network. One skilled in the relevant art will appreciate,however, that the disclosed virtual computer network is illustrative innature and should not be construed as limiting.

Overlay Network Manager

FIG. 1 is a network diagram illustrating an embodiment of an overlaynetwork manager system (ONM) for managing computing nodes associatedwith a virtual computer network. Virtual network communications can beoverlaid on one or more intermediate physical networks in a mannertransparent to the computing nodes. In this example, the ONM systemincludes a system manager module 110 and multiple communication managermodules 109 a, 109 b, 109 c, 109 d, 150 to facilitate the configuringand managing communications on the virtual computer network.

The illustrated example includes an example data center 100 withmultiple physical computing systems operated on behalf of the ONMsystem. The example data center 100 is connected to a global internet135 external to the data center 100. The global internet can provideaccess to one or more computing systems 145 a via private network 140,to one or more other globally accessible data centers 160 that each havemultiple computing systems, and to one or more other computing systems145 b. The global internet 135 can be a publicly accessible network ofnetworks, such as the Internet, and the private network 140 can be anorganization's network that is wholly or partially inaccessible fromcomputing systems external to the private network 140. Computing systems145 b can be home computing systems or mobile computing devices thateach connects directly to the global internet 135 (e.g., via a telephoneline, cable modem, a Digital Subscriber Line (“DSL”), cellular networkor other wireless connection, etc.).

The example data center 100 includes a number of physical computingsystems 105 a-105 d and a Communication Manager module 150 that executeson one or more other computing systems. The example data center furtherincludes a System Manager module 110 that executes on one or morecomputing systems. In this example, each physical computing system 105a-105 d hosts multiple virtual machine computing nodes and includes anassociated virtual machine (“VM”) communication manager module (e.g., aspart of a virtual machine hypervisor monitor for the physical computingsystem). Such VM communications manager modules and VM computing nodesinclude VM Communication Manager module 109 a and virtual machines 107 aon host computing system 105 a, and VM Communication Manager module 109d and virtual machines 107 d on host computing system 105 d.

This illustrative data center 100 further includes multiple physicalnetworking devices, such as switches 115 a-115 b, edge router devices125 a-125 c, and core router devices 130 a-130 c. Switch 115 a is partof a physical sub-network that includes physical computing systems 105a-105 c, and is connected to edge router 125 a. Switch 115 b is part ofa distinct physical sub-network that includes the System Manager module110, and is connected to edge router 125 b. The physical sub-networksestablished by switches 115 a-115 b, in turn, are connected to eachother and other networks (e.g., the global internet 135) via anintermediate communication network 120, which includes the edge routers125 a-125 c and the core routers 130 a-130 c. The edge routers 125 a-125c provide gateways between two or more sub-networks or networks. Forexample, edge router 125 a provides a gateway between the physicalsub-network established by switch 115 a and the interconnection network120, while edge router 125 c provides a gateway between theinterconnection network 120 and global internet 135. The core routers130 a-130 c manage communications within the interconnection network120, such as by routing or otherwise forwarding packets or other datatransmissions as appropriate based on characteristics of such datatransmissions (e.g., header information including source and/ordestination addresses, protocol identifiers, etc.) and/or thecharacteristics of the interconnection network 120 itself (e.g., routesbased on the physical network topology, etc.).

The System Manager module 110 and Communication Manager module 109 canconfigure, authorize, and otherwise manage communications betweenassociated computing nodes, including providing logical networkingfunctionality for one or more virtual computer networks that areprovided using the computing nodes. For example, Communication Managermodule 109 a and 109 c manages associated virtual machine computingnodes 107 a and 107 c and each of the other Communication Managermodules can similarly manage communications for a group of one or moreother associated computing nodes. The Communication Manager modules canconfigure communications between computing nodes so as to overlay avirtual network over one or more intermediate physical networks that areused as a substrate network, such as over the interconnection network120.

Furthermore, a particular virtual network can optionally be extendedbeyond the data center 100, such as to one or more other data centers160 which can be at geographical locations distinct from the first datacenter 100. Such data centers or other geographical locations ofcomputing nodes can be inter-connected in various manners, including viaone or more public networks, via a private connection such as a director VPN connection, or the like. In addition, such data centers can eachinclude one or more other Communication Manager modules that managecommunications for computing systems at that data. In some embodiments,a central Communication Manager module can coordinate and managecommunications among multiple data centers.

Thus, as one illustrative example, one of the virtual machine computingnodes 107 a 1 on computing system 105 a can be part of the same virtuallocal computer network as one of the virtual machine computing nodes 107d 1 on computing system 105 d. The virtual machine 107 a 1 can thendirect an outgoing communication to the destination virtual machinecomputing node 107 d 1, such as by specifying a virtual network addressfor that destination virtual machine computing node. The CommunicationManager module 109 a receives the outgoing communication, and in atleast some embodiments determines whether to authorize the sending ofthe outgoing communication. By filtering unauthorized communications tocomputing nodes, network isolation and security of entities' virtualcomputer networks can be enhanced.

The Communication Manager module 109 a can determine the actual physicalnetwork location corresponding to the destination virtual networkaddress for the communication. For example, the Communication Managermodule 109 a can determine the actual destination network address bydynamically interacting with the System Manager module 110, or can havepreviously determined and stored that information. The CommunicationManager module 109 a then re-headers or otherwise modifies the outgoingcommunication so that it is directed to Communication Manager module 109d using an actual substrate network address.

When Communication Manager module 109 d receives the communication viathe interconnection network 120, it obtains the virtual destinationnetwork address for the communication (e.g., by extracting the virtualdestination network address from the communication), and determines towhich virtual machine computing nodes 107 d the communication isdirected. The Communication Manager module 109 d then re-headers orotherwise modifies the incoming communication so that it is directed tothe destination virtual machine computing node 107 d 1 using anappropriate virtual network address for the virtual computer network,such as by using the sending virtual machine computing node 107 a 1'svirtual network address as the source network address and by using thedestination virtual machine computing node 107 d 1's virtual networkaddress as the destination network address. The Communication Managermodule 109 d then forwards the modified communication to the destinationvirtual machine computing node 107 d 1. In at least some embodiments,before forwarding the incoming communication to the destination virtualmachine, the Communication Manager module 109 d can also performadditional steps related to security.

Further, the Communication Manager modules 109 a and/or 109 c on thehost computing systems 105 a and 105 c can perform additional actionsthat correspond to one or more logical specified router devices lyingbetween computing nodes 107 a 1 and 107 c 1 in the virtual networktopology. For example, the source computing node 107 a 1 can direct apacket to a logical router local to computing node 107 a 1 (e.g., byincluding a virtual hardware address for the logical router in thepacket header), with that first logical router being expected to forwardthe packet to the destination node 107 c 1 via the specified logicalnetwork topology. The source Communication Manager module 109 a receivesor intercepts the packet for the logical first router device and canemulate functionality of some or all of the logical router devices inthe network topology, such as by modifying a TTL (“time to live”) hopvalue for the communication, modifying a virtual destination hardwareaddress, and/or otherwise modify the communication header.Alternatively, some or all the emulation functionality can be performedby the destination Communication Manager module 109 c after it receivesthe packet.

By providing logical networking functionality, the ONM system providesvarious benefits. For example, because the various Communication Managermodules manage the overlay virtual network and can emulate thefunctionality of logical networking devices, in certain embodimentsspecified networking devices do not need to be physically implemented toprovide virtual computer networks, allowing greater flexibility in thedesign of virtual user networks. Additionally, correspondingmodifications to the interconnection network 120 or switches 115 a-115 bare generally not needed to support particular configured networktopologies. Nonetheless, a particular network topology for the virtualcomputer network can be transparently provided to the computing nodesand software programs of a virtual computer network.

Logical/Virtual Networking

FIG. 2 illustrates a more detailed implementation of the ONM system ofFIG. 1 supporting logical networking functionality. The ONM systemincludes more detailed embodiments of the ONM System Manager and ONMCommunication Manager of FIG. 1. In FIG. 2, computing node A is sendinga communication to computing node H, and the actions of the physicallyimplemented modules 210 and 260 and devices of network 250 in actuallysending the communication are shown, as well as emulated actions of thelogical router devices 270 a and 270 b in logically sending thecommunication.

In this example, computing nodes A 205 a and H 255 b are part of asingle virtual computer network for entity Z. However, computing nodescan be configured to be part of two distinct sub-networks of the virtualcomputer network and the logical router devices 270 a and 270 b separatethe computing nodes A and H in the virtual network topology. Forexample, logical router device J 270 a can be a local router device tocomputing node A and logical router device L 270 b can be a local routerdevice to computing node H.

In FIG. 2, computing nodes A 205 a and H 255 b includes hardwareaddresses associated with those computing nodes for the virtual computernetwork, such as virtual hardware addresses that are assigned to thecomputing nodes by the System Manager module 290 and/or theCommunication Manager modules R 210 and S 260. In this example,computing node A has been assigned hardware address “00-05-02-0B-27-44,”and computing node H has been assigned hardware address“00-00-7D-A2-34-11.” In addition, the logical router devices J and Lhave also each been assigned hardware addresses, which in this exampleare “00-01-42-09-88-73” and “00-01-42-CD-11-01,” respectively, as wellas virtual network addresses, which in this example are “10.0.0.1” and“10.1.5.1,” respectively. The System Manager module 290 maintainsprovisioning information 292 that identifies where each computing nodeis actually located and to which entity and/or virtual computer networkthe computing node belongs.

In this example, computing node A 205 a first sends an addressresolution protocol (ARP) message request 222-a for virtual hardwareaddress information, where the message is expected to first pass througha logical device J before being forwarded to computing node H.Accordingly, the ARP message request 222-a includes the virtual networkaddress for logical router J (e.g., “10.0.0.1”) and requests thecorresponding hardware address for logical router J.

Communication Manager module R intercepts the ARP request 222-a, andobtains a hardware address to provide to computing node A as part ofspoofed ARP response message 222-b. The Communication Manager module Rcan determine the hardware address by, for example, looking up varioushardware address information in stored mapping information 212, whichcan cache information about previously received communications.Communication Manager module R can communicate 227 with the SystemManager module 290 to translate the virtual network address for logicalrouter J.

The System Manager module 290 can maintain information 294 related tothe topology and/or components of virtual computer networks and providethat information to Communication Manager modules. The CommunicationManager module R can then store the received information as part ofmapping information 212 for future use. Communication Manager module Rthen provides computing node A with the hardware address correspondingto logical router J as part of response message 222-b. While request222-a and response message 222-b actually physically pass betweencomputing node A and Communication Manager module R, from the standpointof computing node A, its interactions occur with local router device J.

After receiving the response message 222-b, computing node A 205 acreates and initiates the sending of a communication 222-c to computingnode H 255 b. From the standpoint of computing node A, the sentcommunication will be handled as if logical router J 270 a werephysically implemented. For example, logical router J could modify theheader of the communication 265 a and forward the modified communication265 b to logical router L 270 a, which would similarly modify the headerof the communication 265 b and forward the modified communication 265 cto computing node H. However, communication 222-c is actuallyintercepted and handled by Communication Manager module R, whichmodifies the communication as appropriate, and forwards the modifiedcommunication over the interconnection network 250 to computing node Hby communication 232-3. Communication Manager module R and/orCommunication Manager module S may take further actions in this exampleto modify the communication from computing node A to computing node H orvice versa to provide logical networking functionality. For example,Communication Manager module S can provides computing node H with thehardware address corresponding to logical router L as part of responsemessage 247-e by looking up the hardware address in stored mappinginformation 262. In one embodiment, a communication manager or computingnode encapsulates a packet with another header or label where theadditional header specifies the route of the packet. Recipients of thepacket can then read the additional header and direct the packetaccordingly. A communication manager at the end of the route can removethe additional header.

A user or operator can specify various configuration information for avirtual computer network, such as various network topology informationand routing costs associated with the virtual 270 a, 270 b and/orsubstrate network 250. In turn, the ONM System Manager 290 can selectvarious computing nodes for the virtual computer network. In someembodiments, the selection of a computing node can be based at least inpart on a geographical and/or network location of the computing node,such as an absolute location or a relative location to a resource (e.g.,other computing nodes of the same virtual network, storage resources tobe used by the computing node, etc.). In addition, factors used whenselecting a computing node can include: constraints related tocapabilities of a computing node, such as resource-related criteria(e.g., an amount of memory, an amount of processor usage, an amount ofnetwork bandwidth, and/or an amount of disk space), and/or specializedcapabilities available only on a subset of available computing nodes;constraints related to costs, such as based on fees or operating costsassociated with use of particular computing nodes; or the like.

Route Selection on Substrate Network

FIG. 3 illustrates an example embodiment of a substrate network 300having a route manager 336 capable of determining routes for overlaynetworks. The substrate network 300 can be composed of one or moresubstrate components or nodes, such as computing nodes, routing nodes,communication links or the like. In FIG. 3, the substrate network 300includes computing nodes A 302, B 304, C 306, and D 308, which arecapable of simulating various components of one or more associatedoverlay networks. The nodes can be located on the same data center or inmultiple data centers. Computing node A is interconnected to node B vianetwork W 310, node B is connected to node C by network X 312, node C isconnected to node D by network Y 314, and node D is connected to node Aby network Z 316. Networks W, X, Y, and Z can include one or morephysical networking devices, such as routers, switches, or the like, andcan include private or public connections. Components shown in FIG. 3,such as the computing nodes and communication manager modules, canimplement certain of the features of embodiments described above withrespect to FIGS. 1 and 2.

In FIG. 3, nodes A 302, B 304, C 306, and D 308 are associated with arespective Communication Manager module 320, 322, 324, and 326. Thecommunication manager modules can implement certain of the featuresdescribed in the Communication Manager 150, 210, 260 and VMCommunication manager 109 a, 109 b, 109 c, 109 d of FIGS. 1 and 2. Forexample, the Communication Manager module 320 for node A can operate ona hypervisor monitor of the computing node and can direct thecommunication of one or more virtual computing nodes 330, 332, 334 ofnode A. The computing nodes, communication managers and Route Manager336 can be part of the same ONM system. In one embodiment, the computingnodes run the XEN operating system (OS) or similar virtualization OS,with the communication managers operating on domain 0 or the first OSinstance and the virtual computing nodes being domain U or additional OSinstances.

The communication manager modules in FIG. 3 are in communication with aRoute Manager module 336, operating on one or more computing devices,that directs routing for the substrate network 300. In one embodiment,the Route Manager operates as part of the ONM System Manager module 110,290 of FIGS. 1 and 2, with functionally combined into a single module.The Route Manager can be located within a data center or at a regionallevel and direct traffic between data centers. In one embodiment,multiple Route Managers can operate in a distributed manner tocoordinate routing across multiple data centers.

In FIG. 3, two virtual networks are associated with the substratenetwork 300. Virtual network 1 (VN1) has components 338, 340, 342,associated with virtual computing nodes on computing nodes A 302, B 304,and C 306. Virtual network 2 (VN2) has components 344, 346, 348associated with virtual computing nodes on nodes A, C, and D 308.

As the Routing Manager module 336 directs network traffic on thesubstrate network 300, traffic can be directed flexibly and variousnetwork configurations and network costs can be considered. For example,routing paths can be determined based on specified performance levelsfor the virtual networks. In one embodiment, if the user for VN1 isentitled to a higher service level, such as for faster speed (e.g. lowerlatency and/or higher bandwidth), traffic associated with VN1 can berouted on a “fast” path of the substrate network 300. For example, inone embodiment, traffic for “platinum” users is prioritized over trafficfor “gold” and “silver” users, with traffic from “gold” usersprioritized over “silver” users. In one embodiment, at least somepackets of the user with the higher service level are prioritized overpackets of a user with a lower service level, for example, during timesof network congestion. The user may be entitled to a higher levelbecause the user has purchased the higher service level or earned thehigher service level through good behavior, such as by paying bills,complying with the operator's policies and rules, not overusing thenetwork, combinations of the same, or the like.

The Route Manager 336 can store user information or communicate with adata store containing user information in order to determine the targetperformance level for a virtual network. The data store can beimplemented using databases, flat files, or any other type of computerstorage architecture and can include user network configuration, paymentdata, user history, service levels and/or the like. Typically, the RouteManager will have access to node and/or link characteristics for thesubstrate nodes and substrate links collected using various networkmonitoring technologies or routing protocols. The Route Manager can thenselect routes that correspond to a selected performance level for thevirtual network and send these routes to the computing nodes. Forexample, network W 310 and Y 312 can be built on fiber optic lines whilenetwork Y 314 and Z 316 are built on regular copper wire. The RouteManager can receive network metrics data and determine that the opticallines are faster than the copper wires (or an administrator candesignate the optical lines as a faster path). Thus, the Route Manager,in generating a route between node A 302 and node C 306 for “fast” VN1traffic, would select a path going through network W and Y (e.g., pathA-B-C).

In another situation, where the user for VN2 is not entitled to a higherservice level, VN2 traffic from node A 302 to node B 306 can be assignedto a “slow” or default path through network Y 314 and Z 316 (e.g. pathA-D-C). In order to track routing assignments, the Routing Manager canmaintain the routes and/or route association in a data store, such as aRouting Information Base (RIB) or routing table 350. The Route Managercan also track the target performance criteria 351 associated with aparticular virtual network.

In order to direct network traffic on the substrate network 300, theRouting Manager 336 can create forwarding entries for one or more of theCommunication Manager modules 320, 322, 324, 326 that direct how networktraffic is routed by the Communication Manager. The CommunicationManager modules can store those entries in forwarding tables 352, 354,356, or other similar data structure, associated with a CommunicationManager. For example, for VN1, the Route Manager can generate a controlsignal or message, such as a forwarding entry 358, that directs VN1traffic received or generated on node A 302 through network W 310 (onpath A-B-C). Meanwhile, for VN2, the Route Manager can generate acontrol signal or message, such as a forwarding entry 360, which directstraffic received on node A through network Z. The Route Manager can sendthese forwarding entries to the node A Communication Manager 320, whichcan store them on its forwarding table 352. Thus, network trafficassociated with VN1 and VN2, destined for node C 306 received orgenerated on node A can travel by either path A-B-C or path A-D-C basedon the designated performance level for VN1 and VN2.

While the example of FIG. 3 depicts only two virtual networks, the RouteManager 336 can similarly generate and maintain routes for any number ofvirtual networks. Likewise, the substrate network 300 can include anynumber of computing nodes and/or physical network devices. Routes can bedetermined based on multiple performance criteria, such as networkbandwidth, network security, network latency and network reliability.For example, traffic for a virtual network suspected of being used forspamming (e.g. mass advertisement emailing) can be routed throughnetwork filters and scanners in order to reduce spam.

FIGS. 4A and 4B illustrate a virtual network 401 and correspondingsubstrate network 402 where substrate routing is independentlydetermined from virtual routing. FIG. 4A illustrates a virtual networkincluding several virtual network components. Virtual computing nodes I4404 and I5 406 are connected to a logical router 408. The logical routercan implement certain of the features described in the logical router270 a, 270 b of FIG. 2. The logical router is connected to firewalls I1410 and I2 412. The logical router is configured to direct traffic fromI5 to I2 and I4 to I2, as would be the case if I2 were a backupfirewall. The forwarding table associated with logical router 409reflects this traffic configuration. I1 and I2 are connected to a secondrouter 414. The second router is connected to another virtual computingnode, I3 415. Thus, based on the topology and associated forwardingtable of the virtual network 401, traffic from I4 and I5 to I3 passedthrough I2.

FIG. 4B illustrates an example topology of the substrate network 402associated with the virtual network 401. The substrate network includescomputing node A 420, computing node B and a Route Manager 424.Substrate nodes A and B are each associated with a Communication Manager426, 428. Node A is simulating the operation of virtual components I2,I3 and I5 while Node B is simulating the operation of virtual componentson I1 and I4 on their respective virtual machines. The Route Manager canthen use information regarding the assignments of virtual components tocomputing nodes to optimize or otherwise adjust routing tables for thesubstrate network. The Route Manager can receive such information fromthe Communication Managers and/or the System Manager. For example,assuming I1 and I2 are identical virtual firewalls, the Route Managercan determine that because I5 and I2 are located on the same computingnode, while I4 and I1 are located on the other node, virtual networktraffic can be routed from I5 to I2 and from I4 to I1 without leavingthe respective computing node, thus reducing traffic on the network.Such a configuration is reflected in the illustrated forwarding tables430, 432 associated with the Communication Managers. Thus, routes on thesubstrate network can be determined independently of virtual networkroutes.

In some embodiments, the Route Manager 424 or System Manager canoptimize or otherwise improve network traffic using other techniques.For example, with reference to FIGS. 4A and 4B, another instance of I3can be operated on node B 422, in addition to the instance of I3 on nodeA. Thus, virtual network traffic from I5-I2-I3 and I4-I1-I3 can remainon the same computing node without having to send traffic betweencomputing nodes A and B. In one embodiment, substrate traffic can beoptimized or otherwise improved without having different forwardingentries on the substrate and the virtual network. For example, withreference to FIG. 4B, I4 can be moved from computing node B 422 to nodeA 420, thus allowing virtual traffic from I5 and I4 to I2 to remain onthe same computing node. In this way, a user monitoring traffic onlogical router 408 would see that traffic is flowing according theforwarding table in the router, that is, substrate routing istransparent to the user. Other techniques for optimizing traffic bychanging the association of virtual components with virtual machinesand/or duplicating components can also be used.

In some situations, it can be desired that substrate routes reflectroutes specified in the virtual table. For example, the virtual networkuser can wish to control how traffic is routed in the substrate network.However, rather than giving the user access to the substrate network,which could put other users at risk or otherwise compromise security, adata center operator can propagate network configuration or virtualnetwork characteristics specified by the user for the virtual network tothe substrate network. This propagated data can be used in generatingrouting paths in the substrate network, thus allowing the user to affectsubstrate routing without exposing the substrate layer to the user.

Route Selection on Overlay/Virtual Network

FIGS. 5A and 5B illustrate a virtual route selection propagated to thesubstrate network. FIG. 5A illustrates a virtual network topology wherelogical network 1 (LN1) 502 is connected to logical network 2 (LN2) 504and logical network 3 (LN3) 506 by a logical router 508. The currentpreferred routing path specified by the user is from LN1 to LN2.

A user may wish to specify a route for various reasons. For example,routing costs through LN2 can be cheaper than LN3, such as when LN2 andLN3 are in different locations with different ISPs and one ISP chargeslower rates than another. In another example, LN3 can be a backupvirtual network for LN2, and used only in some situations, such as forhandling overflow from LN2.

Referring back to FIG. 5A, the user can specify preferred routes throughthe virtual network and/or characteristics or costs associated with thevirtual components, such as monetary costs, packet loss rates,reliability rate, and/or other metrics. These characteristics can beassigned to the virtual components, such as the virtual computing nodes,node links, logical routers/switches or the like. The Route Manager 510can then determine routing tables 512 and/or forwarding tables 514 forthe virtual network.

FIG. 5B illustrates an example of a substrate route that can correspondto the virtual route in FIG. 5A. In the figure, there are three datacenters 520, 522, 524 corresponding to the logical networks 502, 504,506 of FIG. 5A. In data center 1 (DC1), a computing node 526 isconnected to a network translation device A (NTD A) 528 and a networktranslation device B (NTD B) 530. The network translation devices areconnected to external networks C 532 and D 534, respectively.

The network translation devices can serve as a gateway or entry/exitpoint into the virtual network. In some embodiments, the networktranslation devices can translate between a first addressing protocoland a second addressing protocol. For example, if the virtual network isusing IPv6 and the external networks are using IPv4, the networktranslation devices can translate from one addressing protocol to theother for traffic in either direction. In one embodiment, users connectfrom their private networks to the data centers via a VPN or otherconnection to a network translation device, which translates and/orfilters the traffic between networks.

Referring back to FIG. 5B, network C 532 connects data center 2 522 toNTD A 528. Network D 534 connects data center 3 524 to NTD B 530. TheRoute Manager module 510 is in communication with data center 1 520,data center 2 522, and data center 3 524, particularly with theCommunication Manager for the computing node 526.

From information associated with the virtual network, the Route Manager510 can determine that the user wants to route traffic from LN1 to LN2.The Route Manager can then “favor” substrate routes associated with theLN1 to LN2 virtual path. For example, the Route Manager can specify alow routing cost (e.g. cost 1) for communications, such as data packets,travelling on Network C relative to Network D (e.g. cost 10) such thatduring route determination, routes through Network C are favored. In oneembodiment, the Route Manager can apply a coefficient to storedsubstrate costs in order to favor one route over another. In anotherexample, explicit routing paths can be set up corresponding to thevirtual route. The Route Manager can identify routes in its routingtable and communicate those routes with one or more CommunicationManagers.

Referring back to FIG. 5B, when the computing node 526 receives orgenerates a packet destined for LN2 or a network reachable from LN2, thecomputing node can be configured by the Route Manager to send packetsthrough NTD A 528 as it lies on the route including network C 532.

By propagating virtual network configuration data to the substrate, andusing that configuration data in substrate route calculation, amechanism is provided for a virtual network user to affect substraterouting. In some embodiments, the virtual configuration data can be usedin determining association of the virtual components with the substratecomponents. For example, components of the same virtual network can beassociated with the same substrate computing node or on computing nodesconnected to the same switch in order to minimize or otherwise improvesubstrate network traffic. Configuration data can also be provided theother way and, in some embodiments, the user and/or virtual network canbe provided with additional substrate information, such ascharacteristics of the underlying associated substrate components (e.g.performance, costs) in order to make more informed routing decisions.

FIG. 6 illustrates an example substrate network wherein a networktranslation device determines routes into or out of a virtual network.In FIG. 6, a communication, such as a data packet, leaves computing nodeA, which is associated with a virtual network, through NTD B 604. Thenetwork translation device can include a Route Determination module 605for determining the packet route. NTD B is connected to network C 606and network D 608.

In FIG. 6, the Route Manager 610 receives a network configuration ordetermines that route A-B-C is preferred or has a cheaper cost. TheRoute Manager can store the route in a routing table 612. The RouteManager can then send forwarding entries to the NTD B 604 that configureit to send traffic through network C 606. NTD B can contain multipleforwarding entries for multiple virtual networks, such that data for onevirtual network can be sent through network C, while another virtualnetwork sends data through network D. In some cases, network packetswith the same source and/or destination are sent by different networksbased on the associated virtual network.

In some embodiments, the substrate component may not have aCommunication Manager or a Route Determination module and other ways ofcoordinating routing can be used. For example, a substrate component,such as an ordinary router or a network translation device, can be setup multiply on separate paths. Using blacklists, network traffic for aparticular virtual network can be allowed on one path but blocked onothers. The Route Manager can send a control signal or message updatingthe blacklists to manage the data flow.

In other embodiments, substrate components can implement IP aliasing,where, for example, “fast” path packets use one set of IP addresses,while “slow” path packets use another set of IP addresses. When thesubstrate component receives the packet, it can determine which path touse based on the IP address. The Route Manager can send a control signalor message to assign IP addresses to the components based on the type oftraffic handled.

Other ways of differentiating how packets are handled by substratecomponents include: tagging of packets, such as by Multiprotocol LabelSwitching (MPLS); MAC stacking where a packet could have multiple MACaddresses, the first MAC address for a substrate component, such as aswitch, and a second MAC address for a next component either on the“fast” or the “slow” path; and using Network Address Translation (NAT)devices on both ends of a network in order to redirect traffic into thenetwork, such as by spoofing or altering an destination address for anincoming packing and/or altering an the source address of an outgoingpacket. In some embodiments, the Route Manager generates control signalsor messages for coordinating traffic on the substrate network for thevarious techniques described above.

Virtual Network Route Selection Process

FIG. 7A illustrates a flow diagram for a process 700 of propagatingvirtual routes to a substrate network usable in the example networksdescribed above. The virtual routes can be based on networkconfiguration data provided by a virtual network user, such as costs,component characteristics, preferred routes and/or the like.

At block 705, the Route Manager module receives user configurationand/or network configuration data, such as, for example, policy basedrouting decisions made by the user. In some embodiments, a userinterface is provided, allowing a user to specify configuration data.The Route Manager can receive the configuration data from a data store,for example, if user configuration and/or network configuration data arestored on the data store after being received on the user interface orotherwise generated. In some embodiments, the configuration data caninclude explicit routing paths through the virtual network. In someembodiments, the configuration data can specify associated costs fortraversing components of the virtual network, such as links and/ornodes. These costs can be based on monetary costs, packet loss rates,reliability rate and/or other metrics. These costs can be provided bythe user to configure the virtual network provided by the data centeroperator. However, costs and other network configuration data can comefrom the data center operator themselves in addition to or instead offrom the user. For example, the data center operator can use the virtualnetwork to provide feedback to the user on routing costs, such as byassociating monetary use costs for the substrate computing nodes and/orcomponents. In one example, the data center operator can specify a highcost for a high speed network link or high powered computing node sothat the virtual network user can take into account that cost inconfiguring the virtual network.

At block 710, the Route Manager module determines virtual network routesbased on the user configuration and/or network configuration data. Insome embodiments, routing protocols or the route determinationalgorithms of the routing protocols, such as BGP, OSPF, RIP, EIGRP, orthe like, can be used to determine virtual routes.

At block 715, the Route Manager determines one or more forwardingentries for substrate network components, such as computing nodes,network translation devices, or the like. As the Route Manager candetermine routing paths and propagate routing decisions to the substratecomponents, the Route Manager can coordinate routing within a datacenter and/or between multiple data centers.

At block 720, the Route Manager transmits the forwarding entries to thesubstrate components. At block 725, the substrate component receives theforwarding entries. The substrate network components can store theforwarding entries in FIB tables or similar structures. Generally, aCommunication Manager on the substrate component receives and processesthe forwarding entry and manages communications of the substratecomponent.

However, as discussed above, network traffic can also be coordinated forsubstrate components without a Communication Manager using instead, forexample, a NAT device or the like. In some embodiments, the RouteManager can send blacklist updates, manage tagging of the packets,generate stacked MAC addresses, or the like.

At block 730, the substrate components route packets received orgenerated according to the stored forwarding entries. Generally, aCommunication Manager on the substrate component manages the packetrouting and refers to the forwarding entries to make forwardingdecisions.

Substrate Network Route Selection Process

FIG. 7B illustrates a flow-diagram for a process 750 for determiningsubstrate routing based on target performance characteristics of theassociated virtual network usable in the example networks describedabove. In some instances, the Route Manager can optionally generate avirtual routing table for the virtual network before determiningsubstrate routing. The virtual routing table can be used to determinevirtual routing paths, allowing optimization of network traffic byselective association of the virtual network components with substratecomputing nodes, such as by taking into account physical location andvirtual network traffic patterns. However, generation of the virtualrouting table is not necessary as the substrate routes can be determinedindependently of the virtual routes, as will be described below. Inaddition, user configuration and/or network configuration data providedby the user can be used to describe the virtual network, without needingto generate a virtual routing table.

At block 755, the Route Manager receives characteristics of thesubstrate nodes and/or node links. The Route Manager can receive thecharacteristics data from a data store. In some embodiments, a userinterface is provided, allowing a user to specify characteristics data.The characteristics can describe such things as monetary costs, networkbandwidth, network security, network latency, network reliability and/orthe like. These characteristics can be used in a cost function fordetermining substrate routing paths. This information can be kept by theRoute Manager or data source accessible by the Route Manager.

At block 760, the Route Manager receives a target network performancefor the virtual network. The target performance can be based on apurchased service level by the user, user history, security data or thelike. For example, a service level purchased by a user can have minimumbandwidth, latency or quality of service requirements. In anotherexample, a user can be a new customer with an unknown payment historysuch that the user is provisioned on a “slow” virtual network in orderto minimize incurred expenses in case the user fails to pay. In anotherexample, a user identified as carrying dangerous or prohibited traffic,such as viruses, spam, or the like, can be quarantined to particularsubstrate components. During quarantine, the virtual network componentscan be assigned to specialized substrate components with more robustsecurity features. For example, the substrate components can haveadditional monitoring functionally, such as a deep-packet scanningability, or have limited connectivity from the rest of the substratenetwork.

At block 765, the Route Manager determines substrate network routesbased on the target network performance and/or characteristics of thesubstrate nodes and/or links. In one embodiment, the Route Manager canuse the characteristic data in a cost function for determining routes.Which characteristic to use or what level of service to provide can bedetermined by the performance criteria or target performance. Forexample, for a “fast” route, the Route Manager can use bandwidth and/orlatency data for the substrate network to generate routes that minimizelatency, maximize available bandwidth, and/or otherwise improve networkperformance.

The Route Manager can re-determine routes as needed based on changes inthe network, the configuration data and/or the performance level. Forexample, if a user has purchased N gigabits of “fast” routing but hasreached the limit, the Route Manager can generate new routes and shiftthe user to “slow” routing.

At block 770, the Route Manager transmits forwarding entries for one ormore routes to one or more nodes and/or network translation devices. Insome embodiments, the Route Manager determines forwarding entries forthe substrate components and sends those forwarding entries to thesubstrate components on the path. In some embodiments, the Route Managercan send blacklist updates, manage tagging of data packets and/orgenerate stacked MAC addresses.

At block 775, the Route Manager can optionally update the virtualrouting table based on substrate network routes. By changing the virtualnetwork routing table based on the substrate routes, the virtual networkcan stay logically consistent with the behavior of the substratenetwork. Thus, users will not necessarily be confused by discrepanciesin the virtual routing.

Interaction with a Virtual Network

With reference now to FIGS. 8-10, various embodiments for interactionwith a virtual network will be described. With reference to FIG. 8, asystem 800 for interaction with a virtual network topology simulatedwithin the substrate network 100 of FIG. 1 is illustrated. The virtualnetwork topology includes any number of logical network components 802in connection with a logical gateway 804. The logical components 802 cancorrespond to one or more physical computing devices typicallyassociated with a physical network, such as network devices, serverdevices, storage devices and the like. As described previously, both thelogical gateway 804 and the logical network components 802 may besimulated by physical components of the substrate network 100. In amanner similar to as described above with reference to FIG. 1, thesubstrate network 100 is connected to an external communications network135 which is, in turn, connected to a client computing system 145.Though, for illustrative purposes, a single client computing system 145is shown, one of ordinary skill in the art will appreciate that anynumber of client computing systems 145 may interact with the substratenetwork 100.

The logical gateway 804 may obtain commands from a client computingsystem 145 via the external communications network 135. Illustratively,a client computing device 145 may transmit commands typically associatedwith the management of a physical computing network. In one aspect, theclient computing device commands might correspond to passive reportingcommands that return information regarding the routing of data or thestatus of network components 802. Examples include, but are not limitedto, commands for statistical information regarding the flow of databetween network components 802, commands for information correspondingto the current or past status of the logical gateway 804, or commandsfor retrieval of data transmitted by the logical gateway 804. In anotheraspect, the client computing device requests can correspond to activecommands that modify functional aspects of the logical gateway 804.Examples include, but are not limited to, commands to alter interfacesof the logical gateway 804, commands to shape data flowing throughlogical gateway 804, or commands to alter user configurations stored onthe logical gateway 804. Additionally, though system 800 depicts alogical gateway 804, one of ordinary skill in the art will appreciatethat substrate network 100 may simulate any number of networkingdevices, including routers, multilayer switches, and any networkingdevice capable of interaction with a client computing device 804.Additionally, the substrate network 100 may simulate the functionalityof networking equipment, such as routers, without simulating the networkdevice.

As will be described in more detail below, once a command is received atthe logical gateway 804, a set of processes are determined which areexecuted by physical components of the substrate network. As will bediscussed in more detail below, the set of processes are determined bymapping the received command to the set of processes, such that the userobtains an expected result from the network simulated by the substratenetwork 100. For example, a user may issue a command requesting trafficdata for all traffic from network component 802 a through gateway 804.In the substrate network 100, logical gateway 804 may be simulated by anumber of physical components. For example, traffic from networkcomponents 802 may flow through substrate physical networks 310 and 312of FIG. 3.

In some embodiments, as is described above, traffic which is simulatedto originate from network components 802 may have flow through differentnetworks or network components at different time periods, as isdetermined by the substrate network 100. As such, the requested trafficdata may not exist completely within any individual component of thesubstrate network 100. The logical gateway 804 functions, in part, todetermine which components of the substrate network 100 contain the userrequested data. One this determination is made, the data may becollected from the various physical components, and the information maybe returned to the client computing system 145.

With reference now to FIG. 9, a simplified block diagram of thesubstrate network 100 of FIG. 1 will be described for purposes ofillustrating the interaction between various components of the substratenetwork. However, one skilled in the relevant art will appreciate thatillustrative interaction and communications may include, or otherwiseinvolve, additional components not illustrated in the illustrativedrawing figures.

The substrate network 100 includes a number of physical computingsystems 105 that host one or more virtual machine instances 107. As willbe explained in greater detail, the number of virtual machine instanceshosted on each physical computing system 105 can vary according to thecomputing device resources associated with each individual physicalcomputing system 105 and in accordance with the management policies ofthe substrate network 100. The substrate network 100 also includes avirtual machine manager component, such as ONM system manager 110, formanaging the allocation of virtual machine instances 107 on the variousphysical computing systems 105. Although the virtual machine managercomponent is illustrated with regard to functionality implemented by acomponent of the substrate network 100, in an alternative embodiment,the virtual machine manager component may be implemented as a standalone component of the substrate network, integrated into a singlephysical computing system 105 or distributed as functionalityimplemented among multiple physical computing devices 105.

In communication with the ONM system manager 110 is an interface module904 for obtaining requests from various client computing systems 145 viaan external communication network, such as external communicationsnetwork 135 of FIG. 1. The interface module 904 can obtain variouscommands from the client computing system 145. For example, theinterface module 904 may simulate the client interface to logicalgateway 804 of FIG. 8. The interface module 904 can then facilitateinteraction with the substrate network 100 and the physical computingdevices 105. Illustratively, the interface module 904 may obtaincommands from the client computing systems 145 via multiple interfacemethods, such as simple network management protocol (SNMP) or commandline interface (CLI). Although the interface module 904 is illustratedwith regard to functionality implemented by a component of the substratenetwork 100, in an alternative embodiment, the interface module 904 maybe implemented as a stand alone component of the substrate network,integrated into a single physical computing system 105 or distributed asfunctionality implemented among multiple physical computing devices 105.

In addition, though a single interface module 904 is illustrated, someaspects of the present disclosure include multiple interface modules904. Such interface modules may communicate in order to implementdesired functionality. In some embodiments, an interface module 904 maybe configured to operate under a specific interface or protocol. Forexample, a first interface module 904 may function to receive commandsvia a CLI, while a second interface module may receive commands viaSNMP. As will be appreciated by one of ordinary skill in the art,multiple interface modules 904 may be implemented in a variety ofmanners.

Also in communication with the interface module 904 is one or morestorage nodes 906 for archiving or storing information associated withthe mapping of client commands into processes. The storage nodes 906 cancorrespond to various storage media including physical storage mediaassociated specifically with the substrate network 100. Additionally, oralternatively, the storage nodes 906 can correspond to various networkbased storage networks accessible to the substrate network 110 via acommunication network.

As shown in FIG. 9, the client computing system 145 transmits a commandto substrate network 100. Illustratively, the client may transmit acommand requesting that logical gateway 804 of FIG. 8 redirect allincoming data directed towards network component 802 a of FIG. 8 tonetwork component 802 b of FIG. 8. This request would then be receivedby interface module 904. Secondly, the interface module 904 maps thereceived command to a set of processes using information stored withinthe storage node 906. For example, the storage node 906 may containinformation regarding the virtual machine or machines 107 that functionin the simulation of network components 802 a and 802 b. Illustratively,the simulation of some functions, such as routing functions, may notactually be simulated until a command has been received corresponding tothe requested routing function. In this example, the interface modulemay determine that new routing information must be sent to one or morevirtual machines 107 or their corresponding ONM system managers 110.Third, the interface module causes the execution of the determinedprocesses on the required components of substrate network 100.Illustratively, this may include sending calls to the substrate network100 via established Application Protocol Interfaces (“APIs”) provided bythe substrate network 100. Fourth, the interface module generates a setof results based on the executed processes. In the current example, thismay include responses from an ONM system manager 110 or virtual machines107 that the new routing information has been received and processed. Inthe fifth step, the interface module 904 transmits the generated resultsto the client computing system 145.

With reference now to FIG. 10, a flow diagram illustrative of a virtualnetworking device interface routine 1000 executed by an interfacemodule, such as interface module 904, will be described. Illustratively,routine 1000 can be implemented upon determination that a clientcomputing device 145 wishes to interact with a simulated network device,such as logical gateway 804. At block 1004, the interface module 104obtains a command from a client computing device 145 to log on to thesimulated gateway 804. In some embodiments, this request may be obtainedthrough various protocols, such as via secure shell (SSH). In someembodiments, the simulated network component with which the clientcomputing device 145 wishes to interact may be determined based on thenetwork address, such as an IP address, on which a request is received.In other embodiments, a simulated network component may be identifiedbased on the log on credentials supplied by the client computing device145.

Once a log on request is received, the interface module 904 may, in someembodiments, determine whether the user is authorized to log on. In someembodiments, this determination may not occur until configurations areobtained at block 1006. In other embodiments, information pertaining tolog on requests may be stored separately from client configuration, suchas in a storage node 906. If, based on the information pertaining to logon requests, the interface module determines that the log on request isunauthorized, the interface module may end the networking deviceinterface routine. In these embodiments, if a log on request isdetermined to be authorized, the routine continues at block 1006.

At block 1006, the network configuration, client configuration, andclient permissions are obtained by the interface module 904. Theseobtained items allow the function of the interface module 904 to bealtered according to the desires of the user or the administrator of thesubstrate network 100. For example, the client configuration may includeinformation pertaining to the client's interaction with the interfacemodule 904, such as a preferred style of user interface. Additionally,the client configuration may contain customized commands whichfacilitate specialized interaction with the substrate network 100. Insome embodiments, the client configuration may include other clientdata, such as provided contact information of the client. Clientpermissions may specify which commands or results a client is authorizedto access. In some embodiments, client permissions may specify multipleclients which may access various aspects of the interface module 904according to the permissions accorded to each client. Such clientpermissions might be specified by the operator of the substrate network,or designated users of the virtual network. The network configurationmay contain information regarding the simulated network, the substratenetwork 100, or both. The interface module 904 may use this information,for instance, to determine which functions a client computing device 145may invoke, or to determine which network devices 802 to allowinteraction with. These configurations may be stored in a storage node,such as storage nodes 906. In some embodiments, a client computingdevice 145 may transmit a client configuration or client permissions tothe interface module 904 within the obtained request.

Optionally, at block 1008, a user interface may be caused to bepresented on the client computing device 145. The style of the userinterface may be determined based on the client configuration obtainedat block 1006. For example, a client may prefer to interact with theinterface module 904 via a command line interface. The style ofinterface can be further simulated, such that the interface provided toa user is similar to the interface provided by a specific type ofnetworking device, or similar to a particular network operating system.In some embodiments, a user interface may not be desired or evenrequired, such as where the client is already provided with a userinterface via an application on the client computing device 145.

At block 1010, a command is received from the client computing device145. As discussed above, the user may transmit a command requesting thatlogical gateway 804 redirect all incoming data directed towards networkcomponent 802 a to network component 802 b, or that statistical dataregarding traffic on logical gateway 804 be returned to the user. Insome embodiments, the client command may correspond to a request that asimulated component configure a new interface or protocol. As previouslydiscussed, this command may relate to any command which may be receivedon the networking device which is simulated.

At block 1012, a set of processes are determined based on the commandand the various configurations. As discussed above, these processes maybe determined based on mapping information stored within a storage node,such as storage node 906. For example, the storage node 906 may containinformation regarding to the virtual machines 107 or ONM system manager110 which simulate network components 802 a and 802 b. The storage node906 may further contain information allowing user commands to be mappedinto specific processes. For example, a user command to “show IP routes”may, based on information within the storage node, be mapped to thecommand which would return a routing table of a virtual machine 107 oran ONM system manager 110. Such mapping information may be specific to avirtual machine 107 or ONM system manager 110, such that “show iproutes” may result in an Application Protocol Interface (API) calls whenimplemented by some components, or SNMP commands when implemented byother components. Though API calls and SNMP commands are used forillustrative purposes, the determined set of processes may correspond toany command which may be implemented by a network component. Inaddition, mapping may be associated with the make or model of aparticular component, such that each determined process will carry outthe desired function on a particular component. For example, thedetermined set of processes may depend on the manufacturer or operatingsystem of the components toward which each process is directed.

Therefore, at block 1012, a set of processes would be determined suchthat the routing of incoming data to network component 802 a is redirectto network component 802 b. This may involve modifications to virtualmachines 107 associated with networking components 802 a and 802 b.Additionally, this may involve modifications to one or more ONM systemmanagers 110.

The set of processes may also be determined according to the obtainedclient configuration, network configuration and client permissions. Forexample, a received command may correspond to a customized commandspecified within the client configuration. In some embodiments, clientpermissions may be used to determine whether the user, as determined bythe log on request, is authorized to execute the obtained command. Ifthe interface module 904 determines that the user is unauthorized, theroutine 1000 may be ended. In some embodiments, the user may be notifiedthat they are unauthorized to execute such a command. Further,unauthorized access attempts may be stored in a data log for review byauthorized users or administrators. In this embodiment, if the interfacemodule 904 determines that the user is authorized to execute thecommand, the routine proceeds to block 1014.

At block 1014, the interface module 904 causes the identified set ofprocesses to be executed on components of the substrate network 100. Forexample, where the command specifies that all incoming network trafficdirected towards networking component 802 a be redirected to 802 b, aprocess may be executed on the ONM system managers 110 which correspondto networking components 802 a and 802 b. As described above, this mayinclude sending calls to the substrate network 100 via established APIsprovided by the substrate network 100. In some embodiments, execution ofthe set of processes may be caused by other interface methods, such asSNMP.

At block 1016, the interface module 904 obtains informationcorresponding to the executed processes. For example, a commandrerouting traffic from logical network component 802 a to 802 b mayresult in a notification that the processes executed successfully. Whereother processes are executed, such as requests for traffic statistics,the information obtained may correspond to network information fromvarious ONM system managers 110 or other components of the substratenetwork 100.

At block 1018, a response is generated based on the received informationcorresponding to the executed processes. In some embodiments, this mayinclude simply relaying the information obtained at block 1016. Forinstance, where a command corresponds to a routing request, both theinformation received at block 1016 and the generated response mayindicate that the routing request executed successfully. In someembodiments, the generated requests may differ substantially from theinformation obtained at block 1016. For example, where a commandcorresponds to a request for network traffic, the information obtainedat block 1016 may reflect traffic information of the substrate network100. Because the user may not have knowledge of the topology of thesubstrate network 100, this information may be reformatted in accordancewith the configuration of the simulated virtual network, as may beprovided by the obtained network configuration. This may entailcombining or compiling the information such that it corresponds to anexpected response from a simulated logical component, such as logicalgateway 804. Additionally, a response may be modified by the clientconfiguration or the client permissions. For example, a style ofresponse for some or all commands may be specified in the clientconfiguration. In some embodiments, client permissions may specify thatsome information may not be divulged to a user. In these embodiments,the generated results would omit prohibited information.

At block 1020, the generated response is transmitted to the user. Thoughthis flow diagram is illustrative of a single command and response, oneof ordinary skill in the art will appreciate that the routine, orportions of the routine, may be repeated for multiple commands. Forexample, at block 1020, the routine may return to block 1010 in order toreceive additional client commands. In some embodiments, the routine mayincorporate security features, such that if a command is not received ina specified period of time, the routine may return to block 1004,require a user to provide additional log on criteria.

It will be appreciated by those skilled in the art and others that allof the functions described in this disclosure may be embodied insoftware executed by one or more processors of the disclosed componentsand mobile communication devices. The software may be persistentlystored in any type of non-volatile storage.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art. It willfurther be appreciated that the data and/or components described abovemay be stored on a computer-readable medium and loaded into memory ofthe computing device using a drive mechanism associated with a computerreadable storing the computer executable components such as a CD-ROM,DVD-ROM, or network interface further, the component and/or data can beincluded in a single device or distributed in any manner. Accordingly,general purpose computing devices may be configured to implement theprocesses, algorithms, and methodology of the present disclosure withthe processing and/or execution of the various data and/or componentsdescribed above.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

What is claimed is:
 1. A method of monitoring activity on a virtualnetwork, comprising: receiving a client command, wherein the receivedcommand is directed towards retrieving, from a specified logicalcomponent of the virtual network, information corresponding to aspecified activity of the specified logical component during a specifiedperiod of time, and wherein functionality of the specified logicalcomponent is implemented by one or more components of a substratenetwork without instantiating the specified logical component on thevirtual network; identifying one or more components of the substratenetwork which cause implementation of functionality attributable to thespecified activity during at least a portion of the specified period oftime; causing execution of a set of processes by the identified one ormore components in order to retrieve the information corresponding tothe specified activity, wherein the set of processes are directedtowards receiving activity information from each of the identified oneor more components during at least a portion of the specified period oftime; receiving from the identified one or more components a set ofprocess results responsive to the execution of the set of processes; andcreating, based at least in part on the set of process results, a clientresult in response to the received client command.